Electronic image sensors such as charge coupled device (CCD) image sensors and active pixel sensor (APS) devices are used in many types of electronic imaging systems for generating an electronic representation of a visual image. APS devices, often fabricated in a Complementary Metal Oxide Semiconductor process, are also referred to as CMOS sensors. Typically, these image sensors include a number of light-sensitive pixels (that is, picture elements) arranged in a regular two-dimensional pattern or array of rows and columns, with each individual pixel providing a signal based on the light level of the portion of a scene image projected onto the pixel by a lens.
As a result of ongoing improvements in fabrication and design, CMOS and other APS image sensors may often provide considerably more pixels than are useful for forming an image having reasonably good quality. This is true, for example, for parts of the image that are optically out of focus; no added pixel resolution can compensate for this problem.
Limited depth of field is one consequence of the highly compact packaging that is used for many smaller cameras as well as for cameras integrated into cellphone and other handheld electronic devices. With the fixed-position lenses or lens systems that are used to provide reduced-profile designs for these compact devices, the depth of field of the optical system can be constrained, set to a fixed value.
The advantages of an extended depth of field are well appreciated by those skilled in the optical imaging arts. Extending the depth of field of an image capture system enables a proportionately larger portion of the captured image to have improved resolution and appear to be in focus and can yield not only an image that is inherently more pleasing to the eye of the viewer, but also provides better input for image processing and analysis utilities and improves the performance of a number of image processing applications, such as contrast adjustment, face- or object-recognition utilities, and other applications, for example.
Conventional methods for providing an extended depth of field include techniques such as focus stacking. Focus stacking uses multiple full resolution images of an object taken in succession, each taken at a different focus position of the optical system, over a desired range of focus positions. The images obtained in this manner are then “stacked” or otherwise selectively combined in order to form a composite image that gives each portion of the image the best focus obtained over the range. Description of focus stacking is given, for example, in U.S. Patent Application No. 2005/0286800 entitled “Method and Apparatus for Forming a Multiple Focus Stack Image” by Gouch.
Other approaches for extending the depth of field have applied wavefront coding, which purposefully introduces aberrations in the camera optics, then uses image processing in order to remove them in the final image. Wavefront coding is described, for example, in U.S. Pat. No. 5,748,371 entitled “Extended Depth of Field Optical System” to Cathey et al.
Yet another approach for extending the depth of field uses plenoptic imaging. A plenoptic image capture device captures image data as well as information about the directional distribution of light from the object. One example of a plenoptic device is given in U.S. Patent Application Publication No. 2007/0230944 entitled “Plenoptic Camera” by Georgiev.
Although some measure of increase to depth of field is provided by each of these conventional approaches, there are drawbacks that limit the utility of each approach. Focus stacking, optimized for macro-photography of still objects, generally requires a tripod or equivalent support for minimizing motion artifacts. With a CMOS sensor, a full image must be obtained at each of a number of focal length settings for obtaining the needed stack of images. This solution can be unworkable for most types of consumer photography or casual photography applications. Wavefront coding requires detailed characterization of the point-spread function (psf) of camera optics and is computationally intensive and complex, typically requiring additional circuitry for frame memory and processing, making this an impractical solution, particularly for video image processing. Plenoptic image capture devices also have inherent computational complexity and require component packaging designs that may not be easily adapted to the narrow profile of hand-held devices such as cellphones and the like.
Thus, it can be seen that although methods exist for enhancing or extending the depth of field of an image sensor and its attendant optical system, these methods may be impractical for many types of portable and hand-held imaging apparatus.